-
Notifications
You must be signed in to change notification settings - Fork 13.6k
[asan] Speed up ASan ODR indicator-based checking #100923
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Thank you for submitting a Pull Request (PR) to the LLVM Project! This PR will be automatically labeled and the relevant teams will be If you wish to, you can add reviewers by using the "Reviewers" section on this page. If this is not working for you, it is probably because you do not have write If you have received no comments on your PR for a week, you can request a review If you have further questions, they may be answered by the LLVM GitHub User Guide. You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums. |
@llvm/pr-subscribers-compiler-rt-sanitizer Author: Artem Pianykh (artempyanykh) ChangesSummary: This patch adds an indicator indexed (multi-)map of globals to speed up the search. With a bit of extra memory this massively speeds up indicator based ODR checks. Poisoning based ODR checking remains on the slow path. Internally we see many examples where ODR checking takes seconds (even double digits). With this Notes and considerations:
Test Plan:
Full diff: https://github.com/llvm/llvm-project/pull/100923.diff 1 Files Affected:
diff --git a/compiler-rt/lib/asan/asan_globals.cpp b/compiler-rt/lib/asan/asan_globals.cpp
index d413b1ebc9fc0..26518c8ce4b98 100644
--- a/compiler-rt/lib/asan/asan_globals.cpp
+++ b/compiler-rt/lib/asan/asan_globals.cpp
@@ -21,6 +21,7 @@
#include "asan_suppressions.h"
#include "asan_thread.h"
#include "sanitizer_common/sanitizer_common.h"
+#include "sanitizer_common/sanitizer_dense_map.h"
#include "sanitizer_common/sanitizer_mutex.h"
#include "sanitizer_common/sanitizer_placement_new.h"
#include "sanitizer_common/sanitizer_stackdepot.h"
@@ -35,8 +36,11 @@ struct ListOfGlobals {
ListOfGlobals *next;
};
+typedef DenseMap<uptr, ListOfGlobals *> MapOfGlobals;
+
static Mutex mu_for_globals;
static ListOfGlobals *list_of_all_globals;
+static MapOfGlobals map_of_globals_by_indicator;
static const int kDynamicInitGlobalsInitialCapacity = 512;
struct DynInitGlobal {
@@ -73,6 +77,18 @@ ALWAYS_INLINE void PoisonRedZones(const Global &g) {
const uptr kMinimalDistanceFromAnotherGlobal = 64;
+static void AddGlobalToList(ListOfGlobals *&list, const Global *g) {
+ ListOfGlobals *l = new (GetGlobalLowLevelAllocator()) ListOfGlobals;
+ l->g = g;
+ l->next = list;
+ list = l;
+}
+
+static void AddGlobalToMap(MapOfGlobals &map, const Global *g) {
+ ListOfGlobals *&in_map = map[g->odr_indicator];
+ AddGlobalToList(in_map, g);
+}
+
static bool IsAddressNearGlobal(uptr addr, const __asan_global &g) {
if (addr <= g.beg - kMinimalDistanceFromAnotherGlobal) return false;
if (addr >= g.beg + g.size_with_redzone) return false;
@@ -147,14 +163,20 @@ static void CheckODRViolationViaIndicator(const Global *g) {
*odr_indicator = REGISTERED;
return;
}
+ // Fetch globals with the same ODR indicator
+ auto *relevant_globals_lookup =
+ map_of_globals_by_indicator.find(g->odr_indicator);
+ if (!relevant_globals_lookup) {
+ return;
+ }
+ ListOfGlobals *relevant_globals = relevant_globals_lookup->second;
// If *odr_indicator is DEFINED, some module have already registered
// externally visible symbol with the same name. This is an ODR violation.
- for (ListOfGlobals *l = list_of_all_globals; l; l = l->next) {
- if (g->odr_indicator == l->g->odr_indicator &&
- (flags()->detect_odr_violation >= 2 || g->size != l->g->size) &&
+ for (ListOfGlobals *l = relevant_globals; l; l = l->next) {
+ if ((flags()->detect_odr_violation >= 2 || g->size != l->g->size) &&
!IsODRViolationSuppressed(g->name))
- ReportODRViolation(g, FindRegistrationSite(g),
- l->g, FindRegistrationSite(l->g));
+ ReportODRViolation(g, FindRegistrationSite(g), l->g,
+ FindRegistrationSite(l->g));
}
}
@@ -225,10 +247,13 @@ static void RegisterGlobal(const Global *g) {
}
if (CanPoisonMemory())
PoisonRedZones(*g);
- ListOfGlobals *l = new (GetGlobalLowLevelAllocator()) ListOfGlobals;
- l->g = g;
- l->next = list_of_all_globals;
- list_of_all_globals = l;
+
+ AddGlobalToList(list_of_all_globals, g);
+
+ if (UseODRIndicator(g) && g->odr_indicator != UINTPTR_MAX) {
+ AddGlobalToMap(map_of_globals_by_indicator, g);
+ }
+
if (g->has_dynamic_init) {
if (!dynamic_init_globals) {
dynamic_init_globals = new (GetGlobalLowLevelAllocator()) VectorOfGlobals;
|
Thanks for the patch! |
Thanks for the context @vitalybuka! This has become a bit of a bottleneck for us, hence this patch. Looking forward to hear what you think. |
01847d1
to
bc894e6
Compare
This reverts commit 2a5f7e5.
When ASan checks for ODR violation on a global it loops over a linked list of all globals to find those with the matching value of an indicator. For larger binaries with a ton of globals this gets extremely expensive: O(N^2) iterations + potentially poor memory locality from the linked list. This patch adds an indicator indexed (multi-)map of globals to speed up the search. With a bit of extra memory this massively speeds up indicator based ODR checks. Poisoning based ODR checking remains on the slow path.
[asan] Speed up ASan ODR indicator-based checking When ASan checks for a potential ODR violation on a global it loops over a linked list of all globals to find those with the matching value of an indicator. With the default setting 'detect_odr_violation=1', ASan doesn't report violations on same-size globals but it still has to traverse the list. For larger binaries with a ton of shared libs and globals (and a non-trivial volume of same-sized duplicates) this gets extremely expensive. This patch adds an indicator indexed (multi-)map of globals to speed up the search.
… get unregistered The test demonstrates a FN due to indicator value being shared among all globals with the same indicator address. Moving dlopen/dlclose around and/or adding more shared libs into the mix can show more edge cases but this is out of the scope of this commit.
151257d
to
c5598fa
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Will land in 30min or so.
Thanks for the review @vitalybuka! |
@artempyanykh Congratulations on having your first Pull Request (PR) merged into the LLVM Project! Your changes will be combined with recent changes from other authors, then tested Please check whether problems have been caused by your change specifically, as How to do this, and the rest of the post-merge process, is covered in detail here. If your change does cause a problem, it may be reverted, or you can revert it yourself. If you don't get any reports, no action is required from you. Your changes are working as expected, well done! |
BTW. I have patches for that and will CC you. |
Will investigate soon. The test is from #100923.
Not sure I've seen those prominently in profiles. Will double check. Thanks for the heads up @vitalybuka! |
Just for my own curiosity, but @vitalybuka where/how DenseMap's destructor was causing problems? 54c9404 |
I don't have a particular case, we avoid global destructors in general. |
and constructors |
Summary:
When ASan checks for a potential ODR violation on a global it loops over a linked list of all globals to find those with the matching value of an indicator. With the default setting 'detect_odr_violation=1', ASan doesn't report violations on same-size globals but it still has to traverse the list. For larger binaries with a ton of shared libs and globals (and a non-trivial volume of same-sized duplicates) this gets extremely expensive.
This patch adds an indicator indexed (multi-)map of globals to speed up the search.
Internally we see many examples where ODR checking takes seconds (even double digits). With this patch it's practically free and
__asan_register_globals
doesn't show up prominently in the perf profile anymore.There are several high-level questions:
-fsanitize-address-use-odr-indicator
so I'm not sure if poisoning-based check would exhibit the same behavior (looking at the code, the shape looks very similar, so it might?).-1
need to be skipped for the purposes of ODR checking (cf. a257639). But they are still getting added to the list of globals and hence take up space and slow down the iteration over the list of globals. It would be a good saving if we could avoid adding them to the globals list.Test Plan:
cmake --build build --target check-asan
looks good